simon-5510-08-slides

Topics to be covered

  • What you will learn
    • Random and non-random samples
    • Different types of probability samples
    • Different types of non-probability samples
    • Matching and pairing
    • The methods section

What is a population?

  • A group of people or objects that share one or more common features.
    • Demography
    • Geography
    • Occupation
    • Time
    • Care requirements
    • Diagnosis

What is a sampling frame

  • Physical list
    • Ideally everyone or almost everyone in population
    • Used to draw your sample
  • Expensive, not always available.
  • Example: Master Address File.

What is a sample?

  • A sample is a subset of a population
  • Representativeness more important than size
  • Reasons for sampling
    • Expense
    • Time
    • Quality control

Two major types of samples

  • Random sample
    • Everyone has known non-zero probability
  • Non-random sample
    • Different selection probabilities
    • Some may have zero selection probability

Extreme example: World War II bombers

Image of bomber with indication of damage

Example: in school survey of drug use in teenagers

  • Who has lower selection probability?
  • Who has a zero selection probability?
  • Can you redefine your population?

Example: prisoner IQ study

  • Hypothetical study
    • Calculate average IQ of prisoners
    • Lower than general public
  • Conclude: criminals less intelligent than honest people(???)

Break #1

  • What you have learned
    • Random and non-random samples
  • What’s coming next
    • Different types of probability samples

Sampling

  • Sampling designs – Probability sampling
    • Simple random sampling
    • Systematic sampling
    • Stratified sampling
    • Cluster sampling

How to draw a simple random sample

  1. List the sampling frame in a logical order

  2. Attach a column of random numbers

  3. Sort by the column of random numbers

  4. Select your sample, starting at the top

Simple random sample using Microsoft Excel

A spreadsheet illustrating simple random sampling

How to draw a stratified random sample

  1. List the sampling frame and strata in a logical order

  2. Attach a column of random numbers

  3. Sort by the strata and the column of random numbers

  4. Select your sample, starting at the top

Stratified random sample using Microsoft Excel

A spreadsheet illustrating stratified random sampling

Break #2

  • What you have learned
    • Different types of probability samples
  • What’s coming next
    • Different types of non-probability samples

Break #2

  • What have we learned so far?
    • Types of probability samples
    • How to draw a random sample
  • What is coming up next?
    • Different types of non-probability samples
    • How to allocate treatments randomly

Sampling

  • Sampling designs – Nonprobability sampling
    • Convenience sampling
    • Quota sampling
    • Purposive sampling
    • Purposeful sampling
    • Snowball sampling

Example of a purposive sample

Table describing purposive sampling strategy

Randomizing treatments within a convenience sample

Many studies use a convenience sample, which may hamper external validity, but they randomly assign treatment or control conditions within the convenience sample, which helps with internal validity. The process works much like the process of drawing a simple random sample.

  1. List your treatment groups in a logical order

  2. Attach a column of random numbers

  3. Sort by the column of random numbers

  4. Allocate treatment groups, starting at the top of the list.

Randomizing treatment allocation using Microsoft Excel

A spreadsheet illustrating random treatment allocation

Randomizing a crossover trial using Microsoft Excel

A spreadsheet illustrating random allocation of treatment order

Break #3

  • What you have learned
    • Different types of non-probability samples
  • What’s coming next
    • Matching and pairing

Matching and pairing

  • Improved precision
  • Logistical issues
  • Works for both randomized and observational studies

The logistics of matching

  • Not obvious
  • Simplest solution: greedy matching
  • Unpaired patients are lost to your analysis
    • Extra precision from pairing
    • Loss of precision from loss of the unpaired.

The cross-over trial

  • Only for some randomized trials
  • Each subject serves as own control
  • Randomize treatment order
  • Beware of carry-over

Break #4

  • What you have learned
    • Matching and pairing
  • What’s coming next
    • The methods section

What purpose does a methods section serve?

  • Assessment of the quality of your research
    • Brag here about your rigor
    • Save limitations for discussion
  • Allow others to replicate/extend
    • Non-obvious details

What should not be included in the methods section

  • “The Methods section should include only information that was available at the time the plan or protocol for the study was being written; all information obtained during the study belongs in the Results section.”
    • Uniform requirements for manuscripts submitted to biomedical journals: Writing and editing for biomedical publication. J Pharmacol Pharmacother. 2010;1(1):42–58.
  • Exceptions
    • Patient counts, Dropout rates, Protocol changes

What belongs in the methods section

  • Every methods section is different
  • General structure
    • Participants
    • Materials
    • Procedures
    • Measures
    • Analysis

Participants

  • Where you will find your participants
  • Inclusion/exclusion criteria
  • Efforts to insure representativeness

Materials/Procedures

  • Only document the non-routine
  • Materials
    • Chemicals
    • Include company and location
  • Procedures
    • Running complex equipment
    • Multiple step laboratory methods

Measures

  • Outcome variables
  • Independent variables
  • Covariates
  • Validity/reliability

Analysis

  • Research hypotheses / questions
  • Sample size justification
  • Descriptive methods
    • Boilerplate: “Continuous variables were summarized as means and SDs, and categorical variables were summarized as percentages.” Saleem 2019.

Analysis

  • Statistical model
  • Adjustments for multiplicity
  • Handling missing values/dropout
  • Alpha level and one/two sided tests
    • Boilerplate: “All tests were two sided, and P values below the 5% level were regarded as significant.” Lokken 1995.

What goes in the methods section of a qualitative study

  • Recruitment process
  • Structure of the interview/focus group
  • Recording and transcription details
  • SoftWare used to create categories
  • Process to insure reliability
    • Multiple raters
    • Adjudication of disagreement
    • Other audits

Summary

  • What you have learned
    • Random and non-random samples
    • Different types of probability samples
    • Different types of non-probability samples
    • Matching and pairing
    • The methods section